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Structured abstract: Introduction: Although image description has been iden¬ 
tified as an accommodation for presentations conducted in the classroom, only 
a few U.S. states have approved it for use in high-stakes assessments. This study 
examined the use of audible image description as an assessment accommodation 
for students with visual and print disabilities by investigating student compre¬ 
hension under multiple conditions. Methods: Students in three western states in 
grades three through eight who had visual (n = 117) or print (n = 178) 
disabilities participated in an abbreviated test constructed of retired assessment 
questions in English language arts, mathematics, and science, that were aligned 
with each state’s instructional standards, under conditions with and without 
standardized description of graphic images. The study used a within-subjects 
block design to collect and compare comprehension data under conditions where 
audible image description was both used and not used in an abbreviated test. 
Results: Results indicated that students who read braille were more likely to 
respond correctly under the audible image description condition, and students 
with visual and print disabilities who used print were equally likely to respond 
correctly regardless of condition. Discussion: Braille readers were more likely to 
obtain a correct answer when audible image description accompanied the ques¬ 
tion. Audible image description did not affect the likelihood of a correct response 
from students with print disabilities or students with visual disabilities who read 
print. Implications for practitioners: Audible image description is an accommo¬ 
dation that may help braille readers perform better on tests. Although the 
Partnership for Assessment of Readiness for College and Careers (PARCC) and 
Smarter Balanced consortia are taking steps to include image (or picture) descrip¬ 
tions in their assessment accommodations, teachers may want to develop a standard 
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method for describing images and fa mi liarize their braille readers to the strategy by 
including it in instruction and in classroom tests. Readers are referred to the National 
Center on Accessible Media’s online guidelines for image description. 


Image description, a method of providing 
access to complex images and graphics to 
children with visual and print disabilities, 
is increasingly important in today’s mul¬ 
timedia classrooms. The amount of mate¬ 
rial with elements needing description is 
so pervasive—videos, DVDs, CDs, web 
pages, textbooks, as well as aspects of 
leisure and recreational activities—that 
the U.S. Department of Education has 
taken an active role in stimulating re¬ 
search and practice to make such images 
and graphics accessible to individuals 
with visual and print disabilities. Some of 
the projects that have been supported in¬ 
clude: a cooperative agreement with the 
National Association of the Deaf to create 
the Described and Captioned Media Pro¬ 
gram (DCMP); the Video Description Re¬ 
search and Development Center at Smith- 
Kettlewell Eye Research Institute; and the 
Digital Image and Graphic Resources for 
Accessible Materials Center (DIAGRAM 
Center), part of Beneficent Technology 
(Benetech). There are a number of addi- 
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tional research studies and projects de¬ 
signed to provide real-time remote audio 
description, or description by crowd¬ 
sourcing, but there has not been any re¬ 
search examining audible image descrip¬ 
tion as an accommodation in assessment 
(Abedi & Ewers, 2013). 

Although image description has been 
identified as an accommodation for presen¬ 
tations given in the classroom (Thompson, 
Morse, Sharpe, & Hall, 2005), such de¬ 
scription has only been approved by a few 
U.S. states for use in high-stakes assess¬ 
ments (Christensen, Lazarus, Crone, & 
Thurlow, 2008). The Smarter Balanced 
Assessment Consortium and the Partner¬ 
ship for Assessment of Readiness for Col¬ 
lege and Careers (PARCC) permit image 
description when it is included as an ac¬ 
commodation on a student’s Individual¬ 
ized Education Program (IEP) or 504 plan 
(Measured Progress/ETS Collaborative, 
2012; PARCC, 2014a, 2014b). The PARCC 
and Smarter Balanced consortia largely 
leave picture description up to test admin¬ 
istrators, who can decide whether the 
image is essential to the question and 
whether description is required or not, al¬ 
though it should be noted that PARCC pro¬ 
vides alternative or “alt” text and para¬ 
graphs that describe some of its images. 
Audible image description has the potential 
to control standardized test administration, 
since all students would receive the same 
image description (rather than leaving it to 
individual human readers to describe the 
graphics in their own words); increase 
independent access to graphic content 
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(because students could control the speed of 
the description); and reduce costs in test 
construction (by creating a recording rather 
than a tactile graphic). Previous research on 
use of audio description by students with 
visual impairments has suggested that such 
description can increase comprehension of 
video materials when it is provided for rel¬ 
evant content (Ely et al., 2006). 

The Utah State Office of Education 
(USOE) received an Enhanced Assess¬ 
ment grant from the Office of Elementary 
and Secondary Education in 2010. Work¬ 
ing in partnership with representatives 
from the Colorado Department of Educa¬ 
tion (CDE), the Kansas State Department 
of Education (KSDE), the National Center 
for Accessible Media at WGBH (NCAM), 
and the National Center on Severe and 
Sensory Disabilities (NCSSD) at the Uni¬ 
versity of Northern Colorado, the grant- 
funded project examined the use of audi¬ 
ble image description as an assessment 
accommodation for students with visual 
and print disabilities by investigating stu¬ 
dent comprehension under multiple con¬ 
ditions. A secondary goal was to docu¬ 
ment meaningful and effective practices 
for access to visual and complex images 
within high-stakes assessments. A key 
feature of this project was the application of 
audible image description practices for stu¬ 
dents with print disabilities other than visual 
impairment, who are defined by Thompson 
et al. (2005) as those students who have a 
“difficulty or inability to visually read stan¬ 
dard print because of a physical, sensory, or 
cognitive disability” (p. 25). 

The use of description has greatly ex¬ 
panded due to the passage of the 21st 
Century Communications and Video Ac¬ 
cessibility Act in 2010. The law not only 
requires broadcasters to provide descrip¬ 


tion in their television programming, but 
it requires that digital content provided 
over the Internet must also be accessible 
to individuals with disabilities. In today’s 
typical classroom, digital content offered 
through the Internet is ubiquitous. Ac¬ 
cording to Thompson et al. (2005), “the 
accommodations provided to a student 
must be the same for classroom instruc¬ 
tion, classroom assessments, and district 
and state assessments” (p. 14). Since there 
seems to be some reluctance by states to 
permit audible image description in testing 
situations, in spite of Thompson et al.’s 
axiom, this project sought to determine if 
audible image description provided advan¬ 
tages or disadvantages for students with vi¬ 
sual and print disabilities. Given the adop¬ 
tion of computer-based assessment with the 
advent of PARCC and the Smarter Bal¬ 
anced consortia, determining whether audi¬ 
ble image description of graphic images is 
an unbiased accommodation seems impor¬ 
tant for all students with disabilities. 

This study thus investigated the follow¬ 
ing question: Does audible image descrip¬ 
tion affect the comprehension of students 
with visual or print disabilities when re¬ 
sponding to questions in English language 
arts (ELA), mathematics, or science? 

Methods 

The study used a within-subjects block 
design (illustrated in Table 1) to collect 
and compare comprehension data under 
conditions in which audible image de¬ 
scription was both used and not used in a 
limited test of retired test items with stu¬ 
dents who had visual or print disabilities. 
Print readers received two questions each 
in ELA, mathematics, and science, under 
two conditions, with and without audi¬ 
ble image description. Braille readers, 
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Table 1 

Randomized block design for reading mode and experimental conditions. 


Reader type 

Presentation conditions for each content area 
(ELA, mathematics, science) 

Reading mode 

Description 

Tactile graphics 

Braille readers 

Braille (9 questions) 

Yes 

Yes 



No 

Yes 



Yes 

No 

Print readers 

Print (6 questions) (large print, standard 

Yes 

— 


print, standard print with 

No 



magnification) 




however, were tested under three condi¬ 
tions: with tactile graphics, the standard ac¬ 
commodation; with audible description of 
the graphic image; and with both a tactile 
graphic and audible description of the 
graphic image. The order of conditions and 
content of questions was randomly assigned 
electronically at the time of testing, so that 
each participant received questions in each 
content area under each condition. Cooney, 
Young, Luckner, and Ferrell (2015) recom¬ 
mend the within-subjects design, where 
participants receive all conditions, as a 
method for establishing causal validity be¬ 
cause the participants act as their own con¬ 
trol group. In this study, participants were 
randomly assigned first to each study con¬ 
dition, and then to one of nine or six ques¬ 
tions, depending on whether they were 
braille or print readers. Thus, there were no 
order effects, since each participant was 
tested in a different sequence of conditions, 
content areas, and questions. 

Study participants 
Recruitment 

Student participants were identified through 
their teachers, who were asked to recom¬ 
mend students in grades three through eight 
who had visual or print disabilities and who 
had taken or were eligible for the state’s 


standard large-scale statewide assessment. 
Teachers were solicited through announce¬ 
ments that were distributed to local educa¬ 
tion agency special education directors, 
assessment directors, curriculum directors, 
special education teachers, and parents or 
guardians of children with disabilities in the 
three states. Project announcements, which 
described the project and offered an hono¬ 
rarium, were also distributed to newsletters 
and electronic discussion groups for teach¬ 
ers of students with visual or print disabil¬ 
ities. The project announcement was also 
shared with teachers at training sessions, 
meetings, and conferences. 

After making initial contact with the 
project coordinators, teachers were asked 
to return a consent form and to distribute 
consent forms to the parents or guardians 
of students who met the requirements for 
the study, according to Institutional Review 
Board (IRB) procedures at the University of 
Northern Colorado. Once both teacher and 
parent or guardian consent forms were re¬ 
ceived, test administrators scheduled visits 
with the teacher and students. 

Students 

A total of 295 student participants were 
tested: 178 with print disabilities and 117 
with visual disabilities. Of the students with 
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Table 2 

Number and percentage of student participants, by grade level and state. 


Grade 

Colorado n (%) 

Kansas n (%) 

Utah n (%) 

Total N (%) 

3 

20 (21.05) 

3 (9.09) 

34 (20.36) 

57 (19.32) 

4 

23 (24.21) 

13 (39.39) 

33 (19.76) 

69 (23.39) 

5 

16 (16.84) 

5 (15.15) 

30 (17.96) 

51 (17.29) 

6 

19 (20.00) 

4 (12.12) 

26 (6.06) 

49 (16.61) 

7 

7 (7.37) 

2 (6.06) 

26 (15.57) 

35 (11.86) 

8 

10 (10.53) 

6 (18.18) 

18 (10.80) 

34 (11.53) 

Total 

95 (32.03) 

33 (11.19) 

167 (56.61) 

295 (100.00) 


visual disabilities, 28 were braille readers, 
73 used large print, and 16 used standard 
print. The number of student participants by 
grade level and state is indicated in Table 2. 
The majority of students came from Utah 
(n = 167) and the greatest grade-level par¬ 
ticipation was fourth grade (n = 69). Al¬ 
though every attempt was made to secure 
equal numbers of students in each grade 
from each state, we were unsuccessful. This 
situation necessitated the use of a specific 
statistical procedure described below. Stu¬ 
dent participants were identified by their 
teachers as having either a print or a visual 
disability at the time of testing. The major¬ 
ity of Colorado students (62.11%) had vi¬ 
sual disabilities; in Kansas and Utah, the 
majority of students had print disabil¬ 
ities (60.61% and 73.05%, respectively). 
Slightly less than 5% of all students with 
print disabilities chose large print as their 
reading medium, and almost 14% of stu¬ 
dents with visual disabilities selected stan¬ 
dard print as their reading medium. 
Students with visual disabilities. Teach¬ 
ers of students with visual disabilities re¬ 
ported an array of visual diagnoses for 
their students. The most common diagno¬ 
ses were optic nerve hypoplasia (13.7%) 
and some form of albinism (12.8%). 
Students with print disabilities. “Print 
disability,” although it is referred to in the 


Individuals with Disabilities Education 
Act (2004) in relation to the National 
Instructional Materials Accessibility Standard 
(NIMAS), Individuals with Disabilities 
Education Act (IDEA), §§ 612(a)(23)(B), 
613(a)(6)(B), is defined by the Code of 
Federal Regulations (1981) for the Na¬ 
tional Library Service for the Blind and 
Physically Handicapped (NLS) as: 

(iii) Persons certified by competent au¬ 
thority as unable to read or unable to 
use standard printed material as a re¬ 
sult of physical limitations. 

(iv) Persons certified by competent 
authority as having a reading dis¬ 
ability resulting from organic dys¬ 
function and of sufficient severity 
to prevent their reading printed ma¬ 
terial in a normal manner. (36 C.F.R. 
701.10(b)). 

In this study, the students who were iden¬ 
tified by their teachers as having a print 
disability were overwhelmingly reported 
as having learning disabilities (79.3%). 
Other disabilities reported included au¬ 
tism, speech or language disorders, and a 
variety of physical disabilities such as 
muscular dystrophy and other health 
impairments. 
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Test development 

An essential component of this project 
was the development of an assessment 
that could be used to evaluate the effec¬ 
tiveness of audible image description as 
an accommodation. The process to design 
the assessment included the development 
of a test item bank, bias and sensitivity 
review, and production of the assessment 
in multiple formats (print, large print, 
braille with tactile graphics, and audible 
image description). 

The test item bank for this assessment 
was developed using items selected from 
the Utah Test Item Pool Server (UTIPS). 
UTIPS is an online formative assessment 
tool with non-secure test items. The items 
were developed by active Utah educators in 
cooperation with the Utah State Office of 
Education. Items are aligned to the Utah 
Core Curriculum (now Utah Core Stan¬ 
dards) by standard and indicator, and where 
appropriate by intended learning outcome. 
The item pool consisted of multiple item 
types (such as forced response, essay, 
matching, short answer, grouped questions, 
and performance questions); we selected 
only forced response items so that no inter¬ 
pretation of students’ answers was required. 

The resulting test item bank for this 
project consisted of 340 total items (101 
ELA, 143 mathematics, 96 science). Two 
reviewers evaluated each question for 
face validity, based on how well an indi¬ 
vidual test item measured an identified 
standard. For ELA and mathematics test 
items, the standards were matched to the 
Common Core State Standards (National 
Governors Association, 2010) adopted by 
the three states. Since the Common Core 
State Standards did not include standards 
for science at the time of the study, each 


science question selected corresponded to 
a grade level standard shared by Utah, 
Kansas, and Colorado. 

Three test questions were selected for 
each subject area and grade level, yielding 
54 items. These were then sent to the Col¬ 
orado Visual Bias Committee for review. 
This committee was charged by the Colo¬ 
rado Department of Education with review¬ 
ing statewide test items for questions that 
assumed visual knowledge and were thus 
considered biased against students with vi¬ 
sual impairments. Some of the 54 items 
were edited or deleted because the question 
was outside of the direct experience of stu¬ 
dents with visual impairments, and backup 
questions were substituted. Items were then 
sent to NCAM for the design and scripting 
of the descriptions for the graphic part of 
each test item. Again, some items were re¬ 
jected and replaced based on tactile or de¬ 
scription production needs. Audible image 
descriptions were scripted and recorded by 
NCAM staff members and distributed to 
project staff members for verification and 
review. Final items selected for each grade 
level and subject area were sent to the Utah 
State Instructional Materials Access Center 
(USIMAC) for final production in print, 
braille, braille with tactile graphics, and 
large print. NCSSD staff members con¬ 
ducted a proofreading and editing review of 
the assessment in all formats and requested 
revisions that made the questions consistent 
in terms of formatting and braille code us¬ 
age. Each test question included accompa¬ 
nying digital files with the audible image 
description in human voice. 

Test administration 
Three researchers and three graduate 
research assistants were trained as test 
administrators. The training consisted of 
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reviewing the testing procedures according 
to a script developed by project staff mem¬ 
bers, practicing with the electronic data 
collection tool, and pulling the random¬ 
ized question in the correct medium and 
condition. Test administrators set up ap¬ 
pointments with teacher participants once 
consent forms were received from both 
teachers and parents or guardians, and 
then traveled throughout each state to test 
student participants. 

The test was administered with the 
teacher present, although not always in 
close proximity. After preliminary intro¬ 
ductions, the test administrator asked stu¬ 
dent participants for their assent to partici¬ 
pate in the study, using a script approved by 
the University of Northern Colorado’s IRB; 
when students assented, they signed a form 
containing the assent script and witnessed 
by test administrators. Teacher participants 
completed a student information form while 
the test was being administered. The stu¬ 
dent information form requested the stu¬ 
dent’s disability diagnosis and the type of 
accommodations identified for each stu¬ 
dent, as well as asking the teacher’s opinion 
of each student’s current reading status (on 
grade level or not). 

Randomization and sequence 
of administration 

Simultaneously with the test development 
phase, an external consultant created a web¬ 
site for the collection of data using iPads. 
The website collected demographic data 
about the student being tested (birthdate, 
gender, state, grade, print or visual disabil¬ 
ity, and reading modality), then generated a 
unique identifier that was used for all sub¬ 
sequent data collection for that student. The 
website then presented questions randomly 
within each medium (print, large print, 


braille) by content area (ELA, science, 
mathematics), question number, and condi¬ 
tion (audible image description, tactile 
graphics, tactile graphics plus audible im¬ 
age description). The number of ques¬ 
tions and the conditions for testing were 
determined by the responses to disability 
(print or visual) and reading modality 
(standard print, large print, and braille). 
Student participants received each ques¬ 
tion only once, but the condition and se¬ 
quence varied across students. The test 
administrator played the audible image 
description only when instructed to do so 
by the on-screen program. Student partic¬ 
ipants’ responses were recorded for one 
question before proceeding to the next 
randomized question. Test administrators 
did not know the order of questions and 
conditions until directed by the on-screen 
program. This random assignment proce¬ 
dure was established to control for any 
order effects that might have occurred. 

Practice items 

Because participants, including those with 
visual impairments, had different degrees 
of exposure to audible image description, 
the research team selected six practice 
questions accompanied by description of 
the graphics in each to administer to par¬ 
ticipants before the test protocol was ad¬ 
ministered. The number of practice ques¬ 
tions was reduced to three soon after 
testing began, because students were 
acclimated to the audible image descrip¬ 
tion procedure quickly. 

Data analysis 

The GENMOD procedure in SAS 9.3 was 
used to fit a generalized linear model to the 
categorical repeated measures design (mul¬ 
tiple administrations of the same question 
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Table 3 

Proportion of correct responses by content 
area and reading medium. 


Reading medium 


Content 

area 

Standard 

print 

Large 

print 

Braille 

Total 

ELA 

53.42 

62.50 

52.38 

56.10 

Science 

55.48 

50.00 

46.43 

50.64 

Mathematics 

52.05 

56.25 

40.48 

49.59 

Total 

53.65 

56.25 

46.43 

52.11 


under different conditions). After consult¬ 
ing with a statistician, this procedure was 
determined to best fit the data, because it 
treated each response to each question in 
each condition as a discrete observation and 
would model the average response pattern. 
Students’ results were analyzed by disabil¬ 
ity and medium, collapsing large print and 
standard print into the visual disabilities 
group (and using large print as a variable). 
The results indicated the likelihood of stu¬ 
dents attaining a correct response under 
each condition, content area, and reading 
medium. We followed the recommendation 
of Chinn (2000), using the parameter 
estimate or odds ratio calculated by the 
GENMOD procedure to determine effect 
size. 

Results 

A total of 1,852 questions were adminis¬ 
tered to the 295 participants. Participants 
responded correctly to 52.11% of those 
questions (see Table 3). There were differ¬ 
ences in responses by content area, reading 
medium, and disability group. ELA ques¬ 
tions were answered correctly 56.10% of 
the time, and 50.64% of science and 
49.59% of mathematics questions were an¬ 
swered correctly. Large print readers had 
the highest proportion of correct responses 
(56.25%) across all content areas, followed 


Article 

Table 4 

Proportion of correct responses by content 
area and disability category. 


Content 

area 

Print 

disability 

Visual 
disability, 
reads print 

Visual 
disability, 
reads braille 

ELA 

54.78 

55.06 

52.38 

Science 

43.82 

55.49 

46.43 

Mathematics 

45.48 

52.81 

40.48 

Total 

48.03 

54.20 

46.43 


by standard print readers (53.65%) and 
braille readers (46.43%), who had the low¬ 
est proportion of correct responses. As ex¬ 
plained earlier, some students with print 
disabilities read large print, and some stu¬ 
dents with visual disabilities read standard 
print, so a separate analysis of responses 
was computed by disability group. In this 
analysis (see Table 4), students with print 
disabilities responded correctly to 48.03% 
of the questions. Students with visual dis¬ 
abilities who read print had the greatest 
proportion of correct responses (54.20%), 
and braille readers responded correctly to 
46.43% of the questions. In general, ELA 
questions were answered correctly more of¬ 
ten than science or mathematics questions, 
and students with low vision who used print 
answered a greater proportion of questions 
correctly. 

Students with print disabilities 
For participating students with print disabil¬ 
ities, a total of 1,066 observations contrib¬ 
uted to the analysis. For 512 of these ob¬ 
servations, the student response was 
correct, and for 554 observations, the stu¬ 
dent response was incorrect. The mathemat¬ 
ics questions were selected as the baseline 
(intercept), and the parameters of the anal¬ 
ysis measured the relative effect on the like¬ 
lihood of a correct answer due to the grade 
level, content area (science or ELA), use 
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Table 5 

Analysis of Generalized Estimating Equation (GEE) parameter estimates for students with print 
disabilities. 


Empirical standard error estimates 


Standard 95% confidence 


Parameter 

Estimate 

error 

limits 

Z 

Pr> [Z] 

Effect size 

Intercept (mathematics) 

0.1945 

0.2502 

-0.2958 

0.06849 

0.78 

0.4368 

0.1072 

Grade 

— 0.0586 

0.0407 

-0.1384 

0.0212 

-1.44 

0.1503 

-0.0323 

Large print (LP) 

-0.7487 

0.5881 

-1.9014 

-0.4039 

-1.27 

0.2030 

-0.4128 

Description 

-0.1096 

0.2069 

-0.5152 

0.2959 

-0.53 

0.5963 

-0.0604 

ELA correct 

0.3172 

0.2221 

-0.1181 

0.7525 

1.43 

0.1532 

0.1748 

Science correct 

-0.1560 

0.2183 

-0.5839 

0.2719 

-0.71 

0.4748 

-0.0860 

LP * description 

-0.0154 

0.5341 

-1.0621 

1.0314 

-0.03 

0.9771 

-0.0085 

LP * ELA 

0.3919 

0.7082 

-0.9961 

1.7800 

0.55 

0.5800 

0.2161 

LP * science 

0.8538 

1.0737 

-1.2508 

2.9583 

0.80 

0.4265 

0.4707 

Description * ELA 

0.0875 

0.3120 

-0.5240 

0.6990 

0.28 

0.7791 

0.0482 

Description * science 

0.1103 

0.2908 

-0.4596 

0.6802 

0.38 

0.7044 

0.0608 


Effect sizes were calculated on the odds ratio (parameter estimate) using the formula suggested by 
Chinn (2000). 


of large print, condition (description or 
no-description), and interactions among 
these variables. The results of the GENMOD 
procedure are presented in Table 5. In Table 
5, the Estimate column refers to the likeli¬ 
hood of each parameter, or variable, in pro¬ 
ducing a correct answer. For the intercept- 
baseline (the mathematics questions), 
students with print disabilities were .19 
times more likely to respond correctly. The 
probability (column PR > [Z]) was not sig¬ 
nificant, and the effect size was small, at 
.1072. None of the Z-scores for students 
with print disabilities were significant, indi¬ 
cating that the audible image description 
condition for students with print disabilities 
had no effect on the likelihood of respond¬ 
ing correctly. 

Students with visual disabilities 
Two analyses were conducted for partic¬ 
ipating students with visual disabilities: 
one for those using print (regular and 
large print), and one for those using 
braille. For students with visual disabili¬ 


ties who read print, there were a total of 
534 observations that contributed to the 
analysis. For 289 of these observations, 
the student response was correct, and for 
245 observations the student response 
was incorrect. The mathematics questions 
were again selected as the baseline (inter¬ 
cept), and the parameters of the analysis 
consisted of responses to the science and 
EFA questions; use of large print; pres¬ 
ence of audible image description; grade 
level; and interactions among these vari¬ 
ables. The results are presented in Table 
6. The results indicated that children with 
visual disabilities reading either regular 
or large print were .6437 times less likely 
to provide a correct response in the 
description condition (non-significant; 
E.S. = —.35). None of the Z-scores for 
students with visual disabilities who read 
print were significant, indicating that the 
audible image description condition for 
students with visual disabilities who used 
print had no effect on the likelihood of 
responding correctly. 


©2017 AFB, All Rights Reserved Journal of Visual Impairment & Blindness, July-August 2017 333 









0 CE Article 


Table 6 

Analysis of GEE parameter estimates for students with visual disabilities who read print. 


Empirical standard error estimates 


Parameter 

Estimate 

Standard 

error 

95% confidence 
limits 

Z 

Pr > [Z] 

Effect 

size 

Intercept (mathematics) 

0.6690 

0.5355 

-0.3805 

1.7186 

1.25 

0.2115 

0.3688 

Grade 

— 0.0201 

0.0581 

-0.1340 

0.0937 

-0.35 

0.7288 

-0.0111 

Large print (LP) 

-0.4336 

0.5191 

-1.4510 

0.5838 

-0.84 

0.4036 

-0.2391 

Description 

-0.6437 

0.4550 

-1.5354 

0.2481 

-1.41 

0.1572 

-0.3549 

ELA correct 

0.2692 

0.4801 

-0.6718 

1.2103 

0.56 

0.5750 

0.1484 

Science correct 

-0.1899 

0.5854 

-1.3373 

0.9575 

-0.32 

0.7456 

-0.1047 

LP * description 

0.5619 

0.4130 

-0.2476 

1.3714 

1.36 

0.1737 

0.3098 

LP * English 

-0.2113 

0.5024 

-1.1960 

0.7734 

-0.42 

0.6740 

-0.1165 

LP * science 

0.3964 

0.5914 

-0.7628 

1.5556 

0.67 

0.5027 

0.2185 

Description * ELA 

-0.0058 

0.3956 

-0.7813 - 

0.7696 

-0.01 

-0.9883 

-0.0032 

Description * science 

-0.1360 

0.4345 

-0.9876 

0.7156 

-0.31 

0.7543 

-0.0750 


Effect sizes were calculated on the odds ratio (parameter estimate) using the formula suggested by 
Chinn (2000). 


For students with visual disabilities who 
read braille, there were a total of 252 
observations that contributed to the anal¬ 
ysis—for 117 of these observations, the 
student response was correct, and for 135 
observations, the student response was in¬ 
correct. Mathematics questions in the tac¬ 
tile graphics condition were selected as 
the baseline (see “intercept” in Table 7), 
and the parameters of the analysis con¬ 
sisted of responses to the science and 
ELA questions; presence of description; 
use of audible image description with tactile 
graphics; and interactions among these vari¬ 
ables. Grade level was not included because 
the numbers of braille readers per grade 
level were so small (ranging from three in 
grades five and eight to a maximum of 
seven in grade three). The results are pre¬ 
sented in Table 7. 

The results of this analysis indicated 
that students with visual disabilities who 
read braille were 1.2040 times more likely 
to respond correctly in the audible image 
description condition (p = .0277, E.S. = 
.6638). They were also 1.0594 times more 


likely to respond correctly to ELA ques¬ 
tions {p = .0463, E.S. = .5841). Unexpect¬ 
edly, the odds of responding correctly to 
mathematics questions in the tactile graph¬ 
ics only condition (baseline) were signifi¬ 
cantly poorer (—.9163, p = .0285, E.S. = 
— .5052). None of the other parameters 
were significant. The audible image de¬ 
scription condition for braille readers had a 
significant positive effect on the likelihood 
of responding correctly. 

Discussion 

This study is the first to examine audible 
image description as an accommodation 
in assessment environments, and it in¬ 
volved 295 students in grades three 
through eight in three states. As such, the 
results should be considered preliminary. 

Students with visual disabilities who 
read braille were more likely to obtain a 
correct answer when audible image de¬ 
scription accompanied the question. Sur¬ 
prisingly, mathematics questions in the 
tactile graphics condition (baseline) sig¬ 
nificantly decreased the likelihood of a 
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Table 7 


Analysis of GEE parameter estimates for students with visual disabilities who were braille readers. 


Empirical standard error estimates 

Parameter 

Estimate 

Standard 

error 

95% confidence 
limits 

Z 

Pr > [Z] 

Effect 

size 

Intercept (mathematics 
and tactile graphics) 

-0.9163 

0.4183 

-1.7362 - 

0.0964 

-2.19 

0.0285 

-0.5052 

ELA correct 

1.0594 

0.5317 

0.0173 

2.1015 

1.99 

0.0463 

0.5841 

Science correct 

0.9163 

0.6065 

-0.2725 

2.1050 

1.51 

0.1309 

0.5052 

Description 

1.2040 

0.5468 

0.1323 

2.2756 

2.20 

0.0277 

0.6638 

Description and tactile 
graphics (TG) 

0.3285 

0.6142 

-0.8753 

1.5323 

0.53 

0.5927 

0.1811 

Description * ELA 

-0.7593 

0.6888 

-2.1093 

0.5908 

-1.10 

-0.2703 

-0.4186 

Description * science 

-1.2040 

0.8811 

-2.9309 

0.5230 

-1.37 

0.1718 

-0.6638 

Description and TG * 

ELA 

-0.9069 

0.7115 

-2.3014 

0.4875 

-1.27 

0.2024 

-0.5000 

Description and TG * 
science 

-0.7638 

0.8093 

-2.3500 

0.8223 

-0.94 

0.3453 

-0.4211 


Effect sizes were calculated on the odds ratio (parameter estimate) using the formula suggested by 
Chinn (2000). 


correct response, with a moderate nega¬ 
tive effect size. Drilling down further, we 
found that the proportion of correct re¬ 
sponses in the audible image description 
condition was equal to or greater than the 
tactile graphics condition in every content 
area (see Table 8), even though braille 
readers in grades three through eight un¬ 
doubtedly had the most experience with 
tactile graphics. Although we had ex¬ 
pected to add value by pairing tactile 
graphics and audible image description, it 
was the condition least likely to result in 

Table 8 

Braille readers’ proportion of correct 
responses by condition and content area 
(n = 28 ). 



Content area 


Condition 

Mathematics 

Science 

ELA 

Tactile graphics 
only 

28.6 

50.0 

53.6 

Description only 

57.1 

50.0 

64.3 

Description and 
tactile graphics 

35.7 

39.3 

39.3 


correct responses (less than .40 for each 
content area), perhaps because the ques¬ 
tions presented under this condition al¬ 
ways took longer to answer (a mean 3.6 
minutes for this condition vs. a mean 2.5 
minutes for all questions). 

Students with visual disabilities who 
were print readers had a greater propor¬ 
tion of correct responses when compared 
to braille readers or students with print 
disabilities (see Table 4). Audible image 
description did not impact the number of 
questions these students answered cor¬ 
rectly, nor did it have an impact on the 
number of correct responses given by the 
students with print disabilities. Our con¬ 
clusion is that audible image description 
is an unbiased accommodation during as¬ 
sessments. If it had provided an unfair 
advantage, print readers would have shown 
an effect for the description condition, but 
they did not. 

Of great concern, however, is the finding 
that braille readers were significantly less 
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Table 9 

Number and percentage of readers on grade level. 


Reading medium 

Reading on grade 
level n (%) 

Not reading on 
grade level n (%) 

Total N (%) 

Standard print 

23 (12.78) 

157 (87.22) 

180 (63.29) 

Large print 

51 (62.96) 

30 (37.04) 

81 (28.03) 

Braille 

17 (60.71) 

11 (39.29) 

28 (9.69) 

Total 

91 (31.49) 

198 (68.51) 

289 (100.00) 


likely to respond correctly when tactile 
graphics accompanied the questions 
(see Table 8). The tactile graphics used 
in this study were extensively reviewed 
by experienced teachers of students 
with 

visual impairments, so we feel confident 
that the quality of the tactile graphics was 
not an issue. Yet tactile graphics were par¬ 
ticularly unhelpful for the mathematics 
questions. Audible image description was at 
least as effective as tactile graphics for all 
content areas. This finding raises questions 
about the exposure to tactile graphics of 
braille readers and whether there is a need 
for a systematic curriculum for tactile 
graphics instruction beginning in preschool. 

This study is not without limitations. 
First, the total number of braille readers was 
only 28. The fact that the braille readers 
demonstrated a better chance of respond¬ 
ing correctly with audible image descrip¬ 
tion is important, but it is possible that 
replicating the study on a larger scale 
would yield different results. Since audi¬ 
ble image description had no effect for 
students who read print, whether visually 
impaired or not, a study focusing only on 
braille readers, with and without tactile 
graphics and with and without image de¬ 
scription, might be a better test of the 
audible image description accommoda¬ 
tion itself. 

It is significant to note that teachers iden¬ 


tified over 68% of their students as not 
reading on grade level (see Table 9). Braille 
and large print readers were more likely to 
be reading on grade level (61% and 63%, 
respectively), which may have affected 
their response to the audible image descrip¬ 
tion—that is, the braille and large print 
readers may have been more successful 
readers to begin with, and the audible image 
description easily became a new strategy 
for them. The fact that braille readers’ per¬ 
formance improved with audible image 
description, although other students 
with visual impairments did not improve, 
however, provides even greater support 
for audible image description as an ac¬ 
commodation for braille readers. 

However, the discrepancy between stu¬ 
dents identified as receiving braille as a 
standard accommodation (n = 39) and 
those who actually used braille in this 
study (n = 28) also requires further in¬ 
vestigation. Although it is possible that 
some students were in the process of 
learning to read braille because of a 
deteriorating visual prognosis, it raises 
the question of whether such students 
were receiving braille instruction as di¬ 
rected by their IEPs and guaranteed by 
IDEA. Because access to student records 
would have required a different level of 
confidentiality, we asked teachers to tell us 
which accommodations were permitted for 
instruction and assessment. Future studies 
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that examine assessment accommodations 
for students with visual disabilities may 
want to examine students’ learning media 
assessments, IEPs, and 504 Plans to deter¬ 
mine permitted accommodations. 

There was considerable discussion 
among team members regarding the best 
format to present image descriptions 
(voice or text only vs. navigable text), 
particularly since an earlier NCAM study 
had found that adult braille readers pre¬ 
ferred to control the rate of description 
and the ability to review the description 
themselves (Gould, O’Connell, & Freed, 
2008). The project staff members agreed 
that description presented in a navigable 
text format provided the best opportunity 
to interact with the visual information us¬ 
ing whatever strategies were most effec¬ 
tive for each individual learner. The ben¬ 
efits for older students in particular are 
considerable, since test items often con¬ 
tain complex data that can be better pre¬ 
sented in navigable tables that students 
can explore using their screen readers— 
see the project history and research meth¬ 
odology for the study regarding STEM 
image description reported by NCAM 
(2009b). However, with the compressed 
time line of this project, we could not 
ensure that students with visual impair¬ 
ments had sufficient opportunity to de¬ 
velop skills in navigating tables using 
their screen readers. Nor did we want to 
eliminate potential participants because 
they were unfamiliar with technology. 
After lengthy discussion, we determined 
that the study was more likely to discern 
the impact of audible image description 
as a test accommodation if the research 
design more closely mimicked the current 
practice for all target populations. At the 
time, the least common denominator in all 


three states was paper-based tests. Conse¬ 
quently, image descriptions for this project 
were presented as audio files controlled by 
the test administrator. 

One limitation of this study was the time 
of year when it was administered. Although 
great care was taken to assure that the con¬ 
tent area questions were reflected in each 
state’s grade-level standards, the November 
to February time frame may have meant 
that some students had not yet been ex¬ 
posed to the concepts prior to testing. Re¬ 
peating the study at the end of the school 
year may result in a greater proportion of 
correct and faster response times, but we 
acknowledge that the end of the school year 
is not a very convenient time for teachers or 
students. An alternative approach might be 
to conduct the study at the beginning of the 
school year, but test the students using the 
previous year’s content (for example, use 
the third grade questions for early fourth 
grade students). 

Smarter Balanced and PARCC are com¬ 
mended for including image description in 
the construction of their assessments. How¬ 
ever, there is considerable confusion sur¬ 
rounding their use (sometimes using “im¬ 
age description” and “human reader” 
interchangeably), leaving the decision 
about the relevance of the image to test 
administrators, and requiring that the ac¬ 
commodations used on the tests be speci¬ 
fied in the IEP or 504 Plan. If accommoda¬ 
tions are supposed to level the playing field, 
then all graphics should be accommodated 
in audible description and navigable text 
according to the student’s need and prefer¬ 
ence, without consideration of relevance. 
No image should be left to the test admin¬ 
istrator to decide which elements are impor¬ 
tant or relevant. These kind of decisions 
could result in non-standard administration 
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of the assessments and potentially give 
some students an advantage not available to 
all. 

Conclusion 

Based on the results of this study, audible 
image description is an unbiased accommo¬ 
dation that assisted students who were 
braille readers in responding correctly to 
assessment questions containing graphics. 
Audible image description as an accommo¬ 
dation for braille readers was at least as 
effective as (and sometimes better than) the 
standard tactile graphics accommodation, 
for all content areas. Yet audible image 
description did not significantly affect the 
proportion of correct responses either for 
students with print disabilities or students 
with visual disabilities who were print read¬ 
ers. Particularly because audible image de¬ 
scription offers the ability to standardize the 
description of graphics without giving away 
the correct response, we recommend that it 
be permitted on statewide assessments and 
urge state departments of education to ap¬ 
prove its adoption. We also suggest that the 
national assessment consortia standardize 
image description, so that every student re¬ 
ceives the same content, to maintain the 
integrity of the test. 

This project resulted in written guide¬ 
lines for audible image description, which 
are available from NCAM (2009a). We 
recommend that these guidelines, rather 
than the guidelines written for educa¬ 
tional video and theater productions, be 
used because they specifically address 
print or digital images. NCAM has also 
recently published an excellent guide, 
Item Writer Guidelines for Greater Ac¬ 
cessibility (Freed et al., 2015), based on 
its long history of research and develop¬ 
ment in all forms of description. 
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